Diagnostic decisions of specialist optometrists exposed to ambiguous deep-learning outputs.

Artificial intelligence (AI) has great potential in ophthalmology. We investigated how ambiguous outputs from an AI diagnostic support system (AI-DSS) affected diagnostic responses from optometrists when assessing cases of suspected retinal disease. Thirty optometrists (15 more experienced, 15 less) assessed 30 clinical cases. For ten, participants saw an optical coherence tomography (OCT) scan, basic clinical information and retinal photography ('no AI'). For another ten, they were also given AI-generated OCT-based probabilistic diagnoses ('AI diagnosis'); and for ten, both AI-diagnosis and AI-generated OCT segmentations ('AI diagnosis + segmentation') were provided. Cases were matched across the three types of presentation and were selected to include 40% ambiguous and 20% incorrect AI outputs. Optometrist diagnostic agreement with the predefined reference standard was lowest for 'AI diagnosis + segmentation' (204/300, 68%) compared to 'AI diagnosis' (224/300, 75% p = 0.010), and 'no Al' (242/300, 81%, p = < 0.001). Agreement with AI diagnosis consistent with the reference standard decreased (174/210 vs 199/210, p = 0.003), but participants trusted the AI more (p = 0.029) with segmentations. Practitioner experience did not affect diagnostic responses (p = 0.24). More experienced participants were more confident (p = 0.012) and trusted the AI less (p = 0.038). Our findings also highlight issues around reference standard definition.

Embellishments Revisited: Perceptions of Embellished Visualisations Through the Viewer's Lens.

Embellishments are features commonly used in everyday visualisations which are demonstrated to enhance assimilation and memorability. Despite their popularity, little is known about their impact on enticing readers to explore visualisations. To address this gap, we conducted 18 interviews with a diverse group of participants who were consumers of news media but non-experts in visualisation and design. Participants were shown ten embellished and plain visualisations collected from the news and asked to rank them based on enticement and ease of understanding. Extending prior work, our interview results suggest that visualisations with multiple embellishment types might make a visualisation perceived as more enticing. An important finding from our study is that the widespread of certain embellishments in the media might have made them part of visualisation conventions, making a visualisation appear more objective but less enticing. Based on these findings, we ran a follow-up online user study showing participants variations of the visualisations with multiple embellishments to isolate each embellishment type and investigate its effect. We found that variations with salient embellishments were perceived as more enticing. We argue that to unpack the concept of embellishments; we must consider two factors: embellishment saliency and editorial styles. Our study contributes concept and design considerations to the literature concerned with visualisation design for non-experts in visualisation and design.

The effectiveness of interventions for optometric referrals into the hospital eye service: A review.

PURPOSE: Ophthalmic services are currently under considerable stress; in the UK, ophthalmology departments have the highest number of outpatient appointments of any department within the National Health Service. Recognising the need for intervention, several approaches have been trialled to tackle the high numbers of false-positive referrals initiated in primary care and seen face to face within the hospital eye service (HES). In this mixed-methods narrative synthesis, we explored interventions based on their clinical impact, cost and acceptability to determine whether they are clinically effective, safe and sustainable. A systematic literature search of PubMed, MEDLINE and CINAHL, guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), was used to identify appropriate studies published between December 2001 and December 2022. RECENT FINDINGS: A total of 55 studies were reviewed. Four main interventions were assessed, where two studies covered more than one type: training and guidelines (n = 8), referral filtering schemes (n = 32), asynchronous teleophthalmology (n = 13) and synchronous teleophthalmology (n = 5). All four approaches demonstrated effectiveness for reducing false-positive referrals to the HES. There was sufficient evidence for stakeholder acceptance and cost-effectiveness of referral filtering schemes; however, cost comparisons involved assumptions. Referral filtering and asynchronous teleophthalmology reported moderate levels of false-negative cases (2%-20%), defined as discharged patients requiring HES monitoring. SUMMARY: The effectiveness of interventions varied depending on which outcome and stakeholder was considered. More studies are required to explore stakeholder opinions around all interventions. In order to maximise clinical safety, it may be appropriate to combine more than one approach, such as referral filtering schemes with virtual review of discharged patients to assess the rate of false-negative cases. The implementation of a successful intervention is more complex than a 'one-size-fits-all' approach and there is potential space for newer types of interventions, such as artificial intelligence clinical support systems within the referral pathway.

Assessment of optometrists' referral accuracy and contributing factors: A review.

PURPOSE: In the UK, ophthalmology has the highest number of outpatient appointments within the National Health Service. False-positive referrals from primary care are one of the main factors contributing to the oversubscription of hospital eye services (HESs). We reviewed the accuracy of referrals originating from primary care optometrists and contributing factors, such as condition type and years since registration. RECENT FINDINGS: Of the 31 studies included in the review, 22 were retrospective analyses of referrals and appointments at the HES. Eight were prospective studies, and one used online clinical vignettes. Seven assessed the accuracy of referrals for all ocular conditions. The remaining studies focused on glaucoma (n = 11), cataracts (n = 7), emergency conditions (n = 4), neovascular age-related macular degeneration (n = 1) and paediatric binocular vision (n = 1). The diagnostic agreement for suspected emergency ocular conditions was the lowest, with only 21.1% of referrals considered to require urgent attention in one study. For glaucoma, the first-visit discharge rate was high (16.7%-48%). Optometrist referral accuracy was overall 18.6% higher than General Medical Practitioners'; however, the two mainly referred different ocular conditions. Female optometrists made more false-positive referrals than males (p = 0.008). The proportion of false positives decreased by 6.2% per year since registration (p < 0.001). SUMMARY: There was significant variation in referral accuracy across different ocular conditions, partly due to differences when defining accurate referrals. Optometrists working in primary care are generally more limited in their resources than the HES. Thus, choosing the cautious option of referral when they are unsure could be in the patients' best interests. The possible effect of increased use of advanced imaging on referrals requires evaluation. Although interventions such as refinement schemes have been put in place, these vary across regions, and their approaches such as virtual referral triaging may reduce unnecessary HES face-to-face appointments and promote communication between primary and secondary care.

Data Work: How Energy Advisors and Clients Make IoT Data Accountable.

We present fieldwork findings from the deployment of an interactive sensing system that supports the work of energy advisors who give face-to-face advice to low-income households in the UK. We focus on how the system and the data it produced are articulated in the interactions between professional energy advisors and their clients, and how they collaboratively anticipate, rehearse, and perform data work. In addition to documenting how the system was appropriated in advisory work, we elaborate the 'overhead cost' of building collaborative action into connected devices and sensing systems, and the commensurate need to support discrete workflows and accountability systems to enable the methodical incorporation of the IoT into collaborative action. We contribute an elaboration of the social, collaborative methods of data work relevant to those who seek to design and study collaborative IoT systems.